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1 HIGH-SPEED TRANSMISSION SYSTEM FOR OPTICAL CHANNELS 

CROSS-REFERENCE TO RELATED APPLICATIONS 

This Application claims priority from Provisional Application 60/177,034 entitled HIGH 
5 SPEED TRANSMISSION FOR OPTICAL CHANNELS filed on 1/17/00, which is 
incorporated by reference herein as though set forth in full. 

FIELD OF THE INVENTION 

The present invention relates generally to apparatus and methods for high speed data 
1 0 transmission over optical channels, and, in particular embodiments, to transmission of data using 
pulse amplitude modulation, trellis coding, and equalization techniques. 

BACKGROUND OF THE INVENTION 

The demand for higher capacity data transmission systems continues to increase. To 

1 5 satisfy the ever increasing demand for more data transmission capacity higher baud rate systems 
such as optical channels have been used. At high baud rates some optical fibers may exhibit . 
phenomena such as multimode transmission characteristics and intersymbol interference, which 
can limit the signaling rate available on that fiber. Therefore, there is a need within the art for 
methods and apparatus that are capable of higher baud rates, and for those which can compensate 

20 for the problems encountered with high speed data transmission. 

SUMMARY OF THE INVENTION 

In one aspect of the invention, an apparatus for transmitting data on a fiber optic channel 
is disclosed. The apparatus comprises a trellis encoder that accepts data to be transmitted, applies 
25 a convolution^ coding to a portion of the data, and produces a trellis coding of the data to be 
transmitted. A subset mapper accepts the trellis coding and produces a plurality of pulse 
amplitude modulated (PAM) symbols from the trellis coding. A Tomlinson precoder accepts the 
PAM symbols and applies a Tomlinson precoding the PAM symbols. A converter converts the 
PAM symbols into a form for coupling into a fiber channel. 

30 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other features, aspects, and advantages of the present invention will become 
better understood with regard to the following description, appended claims, and accompanying 
drawings where: 

35 Figure 1 is a block diagram illustrating components of an optical data communication 

system. 

Figure 2 is a block diagram of a optical communication system according to an 
embodiment of the invention. 
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1 Figure 3 is a block diagram illustrating a fiber optic transmitter according to an 

embodiment of the invention. 

Figure 4 is a graphical illustration of Tomlinson-Harashimaprecoding (THP). 
Figure 4a is a graphical illustration of the mapping of excess pulse amplitude modulation 
5 levels which are produced by a Tomlinson-Harashima precoder. 

Figure 5 is a block diagram of a receiver, according to an embodiment of the invention. 
Figure 6 is a simplified block diagram of a receiver, which contains a decision feedback 
equalizer (DFE). • 

Figure 7 is a graphical illustration of the impulse response of an exemplary fiber channel 

10 

Figure 8 is a block diagram illustrating a fiber channel model used in conjunction with a 
laser model based on the rate equations. 

Figure 9 is a graph representing of the impulse response of a linear system having a 
Gaussian impulse response. 
1 5 Figure 1 0 is a graph illustrating an output of a laser model based on the rate equations. 

Figure 1 1 is a graph illustrating an expanded portion of the graph illustrated in Figure 1 1 . 

Figure 12 is a graph of the simulated output of the laser model of Figure 1 0 after passage 
through a simulated channel. 

Figure 13 is an eye diagram of a simulated receiver equalizer, illustrating a transition 
20 between a bilevel training mode and receiving PAM-5 symbols. 

Figure 14 is a graph of a magnified portion of the graph of Figure 13, illustrating the 
convergence of the equalizer during a training sequence. 

25 DETAILED DESCRIPTION OF THE INVENTION 

The demand for higher data carrying capacity transmissions systems continues to increase. 
Accordingly, embodiments of the present invention relate to methods and apparatus for 
increasing the rate of data transmission in optical transmission systems. 

Figure 1 is a block diagram illustrating a prior art optical data communication system. 
30 In Figure 1 a data source 101 provides data to be transmitted over fiberoptic channel 1 09. 

Data source 101 may be, for example, the Internet, a cable television head end, a corporate 
network or a variety of other data sources. 

Data from data source 101 is provided to an encoder 103, which encodes the data. 
Encoding may encompass representing the input data from the data source 101 in a variety of 
35 ways. In the exemplary system of Figure 1 the data encoding comprises translating the data 
received onto a series of OOK (On Off Keying) symbols for transmission using a laser. OOK 
represents the data as a series of on-off pulses or two levels of optical intensity. 

Once the data is encoded, the encoded signal is coupled into an optical channel driver, 
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1 such as a laser driver 1 05, which controls the intensity of a laser 1 07. The output of laser 1 07 is 
coupled into a fiber optic channel 109. The fiber optic channel is further coupled to an optical 
receiver 111. The optical channel 109 may be of various lengths depending upon the application. 

5 The optical receiver 111 accepts the signal provided by the fiber optic channel 109 and 

converts it into an electrical signal. The electrical signal, representing the transmitted data, is 
provided to an OOK data decoder 113, The data decoder 113 reverses the process of the encoder 
103 and recreates the data provided by the data source 101. The data from the decoder may be 
then routed, for example using a data router 1 15, to various user devices. An exemplary user 

1 0 device 117 then receives data, such as video data, from the data router 115. 

Figure 2 is a block diagram of an optical communication system according to an 
embodiment of the invention. Transmitter 200 communicates with receiver 222 over channel 
21 3. In Figure 2 the data to be encoded is coupled into a trellis encoder 20 1 . The trellis encoder 
201 includes convolutional coder 206 and subset mapper 203. The trellis encoder 201 may be 

15 a single trellis encoder or it may be a series of trellis encoders in parallel. 

The outputs of the convolutional coders 206 are further coupled into subset mappers 203 . 
A subset mapper accepts the convolutionally encoded signal and produces multilevel symbols 
205 as an output. The multilevel symbols are then coupled into equalizers 207. The equalizers 
207 are used to compensate for the non-flat response of a channel 203. After equalizing the 

20 multilevel symbols, the equalizers provide the resultant symbols to one or more digital to analog 
converters (D/A) 209. The digital to analog converters 209 accept the equalized multilevel 
symbols, and convert them into analog signals. The analog signals are then coupled serially into 
an optical source such as a laser 211. The digital to analog (D/A) converter(s) provide successive 
signals to laser 2 1 1 during a second time period, and so forth. In other words a signal from a first 

25 D/A converter may be provided to the laser 211 during a first time period, then a signal from a 
second D/A converter may next be provided to the laser 211. In such a manner multiple symbols 
from multiple data sources may be transmitted by the single laser 211. Alternatively, a single 
D/A converter may accept successive values from multiple data sources, converting them into 
a series of analog values to be used to modulate the intensity of the laser output 211. 

30 The output of the laser 211, modulated by the analog representation of the multilevel 

symbols, is coupled into the optical channel 213. The optical channel 213 transmits the intensity 
modulated laser signal to an optical-to-electrical converter 215. The optical-to-electrical 
converter 2 1 5 accepts the optical signal from the channel 2 1 3 and converts it back to an intensity 
modulated series of electrical signals. The optical-to-electrical converter 215 then provides the 

35 amplitude-modulated signals to one or more analog-to-digital (A/D) converters 217. The A/D 
converters converts the series of analog signals to digital signals. The digital signals are provided 
to one or more trellis decoders 219 where the trellis-encoded digital signals are decoded. The 
output of the trellis encoders are provided to a physical coding sublayer (PCS) unit 221 . A 
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1 physical coding sublayer (PCS) may provide bit manipulation, such as decoding, to the signals 
decoded by the trellis decoder 219. The data output of the PCS 221 is then provided to a user 
interface such as an XGMII (extended Gigabit Media Independent Interface), 

Illustratively, the optical communication system depicted in Figure 2 has particular 

5 characteristics. For example, the channel 21 5 is considered to be a standard 62.5/125 \im fiber. 
Fiber is commonly specified in terms of a bandwidth times length product. For a 62.5/125 n 
meter fiber a typical bandwidth times length product is 500 MHz/km. 500 meters of such fiber 
would typically yield a 1 gigahertz bandwidth for a laser wavelength of 1 3 1 0 nanometers. 

To transmit a signal at or less than the Nyquist rate, the minimum bandwidth must equal 

1 0 one half of the symbol rate of the channel. The Nyquist bandwidth of a channel is the maximum 
rate at which signaling can occur on that channel without intersymbol interference (I SI). In other 
words, a system cannot transmit signals faster than the Nyquist rate without intersymbol 
interference. However, an equalizer such as 207 can be used to remove intersymbol interference. 
Equalization in Figure 2 is shown within the transmitter 200. Such equalization is called transmit 

15 side equalization. Equivalently, equalization may be applied at the receiver 223, and either 
equalization may be non-linear. For example, decision feedback equalization (DFE) may be used 
at the receiver 223 . Although the equalization can be done equivalently on the transmitter as well 
as the receiver side, there are certain advantages to placing the equalizer in the transmit side. For 
example, if the equalization is placed within the receiving side, the trellis decoder and an 

20 equalizer must function concurrently. Concurrent trellis decoding and equalization is a 
complication within the receiver that can be avoided by having the equalization circuit in the 
transmitter. It is difficult to combine an equalizer and a trellis decoder, in a receiver, because 
such a receiver would have to decode the trellis while attempting to compensate for the 
intersymbol interference. If the equalization is done in the transmitter, there is no necessity to 

25 compensate for intersymbol interference while decoding the trellis coding. 

Embodiments of the present invention may include, for example, a single trellis encoder, 
a single symbol mapper, a single equalizer etc. Alternatively, the same components may be 
replicated multiple times the signals time multiplexed from such parallel components in order 
to couple them in and out of a single fiber channel. To simplify the disclosure, however, the 

30 components will be illustrated as single components. Those skilled in the art will realize that the 
same components may be used in a variety of parallel configurations. 

For the purposes of example, the multilevel symbols 205 are considered to be part of a 
PAM-5 (pulse amplitude modulation - 5 level) alphabet. 

Figure 3 is a block diagram illustrating the fiber optic transmitter 200 according to an 

35 embodiment of the current invention. Detail of the transmitter 200 is illustrated in Figure 3. The 
trellis encoder 323 accepts a group of R bits from the data source 202. The trellis encoder 305. 
is a rate M/(M+1) convolutional coder of the R bits which are input to the rate m/(MH) encoder. 
R-M bits will be unencoded and M bits will be encoded. The output of the convolutional coder 
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1 305 comprises (M+l) bits. The R-M unencoded bits and the M + 1 coded bits, which are output 
from the convolutional coder 305 are provided to a subset mapper 307. The subset mapper 307 
maps the received bits into a series of multilevel symbols 309, for example, PAM 5. The 
combination of convolutional coder 3 05 and the R-M unencoded bits comprises a trellis encoder 

5 323. The pulse amplitude modulated signals A L through A N have 5 levels, but may have any 
number of amplitude levels, depending on the pulse amplitude modulation scheme chosen. 

The fiber optic channel, as discussed above, illustratively exhibits multirnode transmission 
characteristics at any Nyquist bandwidth of 1 GHZ. Accordingly, the bandwidth available in the 
channel is smaller than required to signal without intersymbol interference (ISI) at a 1 0 GHZ rate. 

10 To achieve the 10 GHZ signaling rate, the channel operates in the presence of intersymbol 
interference. One way to compensate for intersymbol interference is to use an equalizer in the 
receiver. For example, a decision feedback equalizer (DFE) may be used. The DFE is discussed 
in the receiver section. 

A further way to. compensate for the effect of intersymbol interference is to use a 

1 5 Tomlinson precoder 311. 

In Figure 3 multilevel symbols 309 are provided to a Tomlinson Precoder. for example 
in a 10 gigabit per second (GPS) transmission system implemented using a five level pulse 
amplitude modulation - 5 level (PAM-5) transmission scheme. The baud rate necessary to 
achieve a 10 GPS transmission is reduced to five gigabaud because each PAM-5 symbol can 

20 represent five different values. 

There are multiple advantages to reducing the baud rate by using pulse amplitude 
modulation. One advantage is that the system can operate over multirnode and limited bandwidth 
channels over greater distances than would be possible if on/off keying (OOK) were used. 
Another advantage, of using PAM instead of OOK, is that the PAM symbols can represent 

25 multiple bits of information. Accordingly, the speed of the electronic circuits needed to create 
the transmitted signal at the transmitter is reduced. Consequently, the speed of the electric 
circuits needed at the receiver is also reduced. By reducing the required speed of the electronic 
circuits, technology such as CMOS (complimentary metal oxide semiconductor) may be used to 
implement the electronic circuitry. In contrast, high speed electronic circuits can often require 

30 expensive high speed technology such as gallium arsenide or indium phosphide. Because of the 
higher levels of integration presently available using CMOS, a greater level of integration is 
possible than with such technologies as gallium arsenide or indium phosphide, and so it is 
advantageous to use PAM symbols to decrease the signaling rate while keeping the baud rate 
constant. 

35 A potential disadvantage of using multilevel encoding such as PAM-5, instead of the more 

traditional on/off keying, is that a higher signal to noise ratio (SNR) may be required of the 
channel since OOK needs to represent only two levels whereas PAM symbols are multiple levels. 
By using multiple levels the distance between levels is reduced, over using two levels. Because 
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1 the distance between levels is reduced the available noise margin is also reduced. To reduce the 
required signal to noise ratio to a level equivalent to the OOK system, PAM-5 modulation may 
be combined with trellis coding as illustrated in the embodiment of the invention illustrated in 
Figure 3. 

5 In Figure 3 a Tomlinson Harashima (Tomlinson) precoder 3 1 1 functions as a transmit side 

equalizer. A traditional equalizer in a receiver compensates for distortion and uneven frequency 
response caused by the transmission channel. Including the fiber on laser equalization, however, 
can be done equivalently in a receiver or a transmitter. In either case the result is a signal 
characteristic that, when combined with the channel characteristic, ideally results in a flat overall 

1 0 response. 

A problem with attempting to equalize a fiber channel is that the fiber may exhibit nulls. 
In other words, the fiber channel transfer function permits very little signal transmission at a 
particular frequency. An equalizer, attempting to compensate for such nulls, may require an high 
gain to make up for the poor response of the channel. High gain may produce an unstable 
1 5 response in the equalizer. 

Tomlinson Harashima precoding may be used to compensate for frequency nulls within 
the channel. The output of the Tomlinson precoder 3 1 1 is provided to a Digital Analog (D/A) 
converter 313. The D/A converter produces an analog signal, which is used to modulate the 

< 

intensity of the laser 211. The Tomlinson precoder is shown as a simplified representation in ~" 
20 311. The Tomlinson precoder 3 1 1 may actually be a group of Tomlinson precoders, each of 
which operates on one PAM-5 symbol The PAM-5 symbols thus generated are multiplexed into 
the D/A converter 313. 

Figure 4 is a graphical illustration of Tomlinson-Harashima precoding (THP). In Figure 
4, Channel 41 1 A, which is identical to Channel 41 IB, can be described by a Z transform. The 
25 characteristics of Channel 41 1 A and 41 IB can be described by the expression "1+D(Z). M The 
term "D(Z)" term is responsible for the intersymbol interference exhibited by the channel. If it 
is desired to compensate for the characteristic of the channel within the transmitter, a filter with 
a transfer characteristic inverse of the channel must be added. By implementing a feedback loop 
comprising the summation unit 405 A and the feedback filter 407A, a transfer characteristic of 
30 1/(1+D(Z)) is created. The total response would then be 1/(RD(Z)) times 1+D(Z) resulting in 
a net channel characteristic of one, which is the combined response of the precoder and the 
channel. The combined response of the precoder and the channel is therefore a flat response 
which does not introduce any dispersion, and therefore, the signal at the receiver 41 3 A is 
. equalized. 

35 -Difficulties can be encountered because at frequencies where the channel has a lot of 

attenuation, the precoder will have a lot of gain to compensate for the attenuation. At such 
frequencies, the precoder may become unstable. Therefore, in order to stabilize the precoder and 
to limit the amplitude of the signal out of the precoder, a signal V n represented by arrow 403 is 
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1 added to the summation unit 405A. V n = x M where M is the number of levels being 
transmitted on the channel, In the present embodiment, which uses PAM-5, M has a value of 5. 
M is essentially the maximum number of levels desired at the output. Once the output signal of 
the precoder, V n is computed, if the signal exceeds certain limits then V n is subtracted from the 

5 signal Y n and K„ is the smallest integer that brings the output Y n back into the desired range. 
There is always a value for K,, that will meet this condition. M is essentially the maximum 
allowable range of the output of the precoder. Depending on the value of Y n there is a unique 
integer value that will bring the output of the precoder back within the range M. This is the 
basis of Tomlinson Harashima Precoding (THP). In other words, the THP does the inverse 

1 0 channel characteristic filtering then modifies the input to the summation unit by adding an 
integral multiple of M which at the output is bounded. The signal V n is added to the input The 
output of the channel sees a quantity equal to X n plus V n . In other words, the number of levels 
appearing at the receiver has been expanded. Therefore, the slicer in the receiver must be able 
to distinguish X n + V n levels instead of just being able to distinguish X n levels. One price for 

1 5 doing this type of equalization is the increase of the number of levels in the constellation at the 
receiver. Therefore, all that needs to be done in the receiver to recover the original PAM-5 levels 
is to implement a wrap-around scheme such that the excess levels are wrapped around into the 
original PAM-5 levels. The wrap around is illustrated in Figure 4A, 

Figure 4 A is a graphical illustration of the mapping of excess pulse amplitude modulation 

20 levels produced by a THP, such as illustrated in Figure 4. In Figure 4A, 9 PAM constellation 
levels are present. The correct PAM-5 levels, that is 435, 437, 439, 441 and 443 are present. In 
addition, levels 431 and 433 which are higher than the highest PAM level 445, are present 
Additionally, levels 445 and 447 are present which are lower than the lowest level 443 are 
present. In order to map the 9 levels back into the original 5 levels, a modulo type add or subtract 

25 is done. In other words, if the level out to be remapped are above the maximum levels, a modulo 
5 value is subtracted. In the present case, 5 levels are subtracted from level 433, and accordingly 
level 433 maps into level 443. Similarly, 5 levels are subtracted from level 43 1 and level 43 1 
maps into level 441. Similarly level 445, which is below the lowest level of 443, has 5 levels 
added to it and level 445 is thereby mapped into level 435. Similarly, level 447, which is below 

30 the lowest level of 443, has 5 levels added to it and is thereby remapped into level 437. If for 
example 6 levels were present above level 435 then 1 0 would be subtracted from the 6 th level and 
the 6 th level above 435 would map into level 443. 

Referring to Figure 4 precoder 400B illustrates an alternate method of adding the 
correction factor V n to the precoder. In precoder 400A signal Y n is examined and then a value 

35 for K n is decided on and the value V n is then added or subtracted in the summation unit 405A. 
In reality the operation of creating the adjustment value V n is nothing more than a wrap around 
operation. The operation is comparable to the overflow in an accumulator. Therefore, in the 
precoder represented in 400B modulo block 417 is added. In the modulo M block 417, a wrap 
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1 around operation is added to the precoder circuit and automatically accomplishes the adjustment 
accomplished by the summation fo the V n signal in summation block 405A in precoder 400A. 

One problem with Tomlinson precoding is that the number of levels in the receiver may 

5 grow depending on the channel characteristics. For example, in a PAM-5 system the number of 
levels may grow to 1 5 5 20 or more. The proliferation of levels in the receiver may be a problem 
because within the receiver is an A/D converter sampling the multiple received levels. In order 
to accommodate multiple levels, the resolution of the A/D converter can be adversely impacted 
because the levels must be scaled so that they fit within the range of the A/D. That is, the 

1 0 resolution of an A/D converter chscriminating between 5 levels is much better than the resolution 
of that same A/D converter discriminating 1 5 levels. Therefore, adding levels can require a better 
A/D converter, that is one having more bits of resolution. It is therefore desirable to limit the 
number of levels presented to the receiver. The number of levels that are presented to the 
receiver can be accomplished by limiting the value of K^, in precoder 400 A. Such limiting may 

1 5 lead to certain points falling outside of the allowed levels, however in return for the points falling 
outside of the allowed levels, the number of levels presented to the receiver can be limited. This 
form of THP is referred to as Dynamics Limited Precoding (DLP). 

Precoding has several advantages over receiver side equalization. One advantage is that 
a precoder lends itself to a better parallel implementation than receive side equalization such as 

20 Decision Feedback Equalization (DFE). Another advantage of receiver side equalization is that 
when using trellis-coded modulation, precoding allows the trellis decoder to be substantially 
simplified, since the decoder then does not have to deal with intersymbol interference. The 
combination of precoding with trellis coded modulation can approach the Shannon bound for 
channel capacity when good modulation codes are used. Therefore, the present architecture can 

25 provide a close to optimal architecture. 

Figure 5 is a block diagram of a receiver, according to an embodiment of the invention, 
illustrating the decoding of multiple signals transmitted across the same channel. In Figure 5, 
a photo detector 501 accepts a pulse amplitude modulated signal from the fiber optic channel 
109. The photo detector 501 then provides a voltage signal, representative of the signal received 

30 from the fiber optic channel 109, to a pre-amplifier 503. The pre-amplifier 503 amplifies the 
signal provided by the photo detector 50 1 to a suitable level. Pre-amplifier 503 then provides the 
amplified signal to a high pass filter 505. 

High pass filter 505 functions to prevent aphenomenon known as baseline wander. High 
pass filtering the input signal blocks low frequencies thus minimizing low frequency excursions. 

35 Photo detector 501, pre-amplifier 503 and high pass filter 505 generally define the optical 
receiver 111. The boundaries, however, between the optical receiver and decoder are somewhat 
arbitrary and other sources may define boundary line between these blocks differently. 

The output of high pass filter 505 is provided to a programmable gain amplifier (PGA) 
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1 507. The gain of the PGA 507 is controlled by an automatic gain control (AGC) circuit 508. 
AGC circuit 508 controls the gain of the amplifier 507 according to signal levels at the output 
of the retiming block 511. The signal at the output of the PGA 507 comprises a series of high 
speed pulse amplitude modulated voltage signals. The output of the programmable gain 

5 amplifier 507 is coupled into a plurality of interleaved analog to digital converters 509. 

The analog to digital converters (A/D) 509 are timed by a clock provided by the timing 
recovery circuit 515. Each A/D converter, however, receives its own phase of the clock in order 
to sample successive values using successive A/D converters. Because the values received by 
the A/D converters are sampled using a clock having different phases, retiming of the signals is 

1 0 necessary in order to create a synchronized parallel value. The retiming of the A/D samples takes 
place in retiming block 511. Retiming block 511 essentially comprises a clocked register circuit 
or equivalent. By interleaving N A/D converters in the analog to digital block 509, the clock rate 
of each individual converter can be reduced by a factor of N (over the use of a single converter). 
Without the interleaving of analog to digital converters 509 it may be difficult or impossible to 

1 5 fabricate an analog to digital converter, which could sample the input at a high enough rate, in 
order not to lose any successive values in the input data stream. By interleaving the A/D 
converters the necessity of using very high speed circuit technologies, such as gallium arsenido 
or indium phosphide may be avoided. 

Parallel values from the retiming block 511 areprovided to a fine AGC module 513. The 

20 digital values of the synchronized parallel data can be examined in the fine AGC modules to 
determine whether the amplitude of the input signal is correct and to provide fine adjustments 
to the signal values. The timing recovery block 515 may adjust the timing of the analog to digital 
converters. 

The output of the fine AGC block 5 13 is coupled into an N-dimensional trellis decoder 
25 519. An N-dimensional trellis decoder includes N trellis decoders. The number of trellis 
decoders will vary depending on a variety of implementation details. The N-dimensional trellis 
decoder 519 decodes the symbols accepted from the fine AGC module 513 and converts them 
into digital data values. 

Once the PAM-5 symbols have been decoded into bit patterns, they are provided to a 
30 physical coding sublayer (PCS) 521. The physical coding sublayer 521 provides bit 
manipulation, such as signal descrambling, etc. The physical coding sublayer 52 1 then provides 
a resultant bit stream to a system interface 523. 

Figure 6 is a simplified block diagram of a receiver such as that illustrated in Figure 5 
illustrating the addition of a DFE. The DFE in the receiver may be used instead of a Tomlinson 
35 precoder in the transmitter. Because equalization can be done equivalently in the receiver or 
transmitter the net equalization effect is the same. 

The block diagram of Figure 6 does not illustrate the parallelism of the receiver illustrated 
by Figure 5. Figure 6, however does provide detail on the addition of equalization, which may 
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1 be included in the receiver. A photonic signal is accepted by the detector 601 . The detector 60 1 
converts the received signal into a voltage and then provides the voltage signal to pre-amplifier 
603. Pre-amplifier 603 amplifies the signal and provides it to high pass filter 605. A 
programmable gain amplifier 607 accepts the signal from the high pass filter 605 and provides 

5 it to an A/D converter 609. 

The A/D converter 609 converts the analog signal from the high pass filter into a digital 
equivalent. An AGC block 608 accepts the digital value from the A/D converter 609 and 
controls the gain of the programmable gain amplifier (PGA) 607. The output of the analog to 
digital converter 609 is also provided to a fine AGC 613, where small adjustments in the signal 

10 are made. The output of the fine AGC 6 1 3 is then provided to a decision feedback equalizer 625 . 
In order to explain the operation of a decision feedback equalizer reference will be made to 
Figure 7. 

Figure 7 is a graphical illustration of the impulse response of an exemplary fiber channel. 
In Figure 7, point 719 represents a decision point where the value of the waveform 729 is 
1 5 sampled. Since the input waveform is an impulse, by definition only one value (represented by 
point 719) is provided to the channel. Therefore any subsequent response such as values 721, 
723, 725 or 727 do not represent valid values which have been provided to the channel. Values 
72 1 , 723, 725 and 727 instead represent intersymbol interference caused by the impulse function. 

i 

Samples 721 , 723, 725 and 727 may be caused by the dispersion of the impulse waveform within 

20 a multimode fiber and are an undesirable feature of the fiber channel. They are generally caused 
by the differing propagation times of the impulse through different modes of the fiber. An 
equalizer, such as the illustrated DFE 625 may compensate for the distortion introduced by 
samples such as 721, 723, 725 and 727. The decision feedback equalizer uses an adaptive 
transversal filter 71 1 to generate a waveform equivalent to the trailing edge 73 1 of the impulse 

25 response. The trailing edge, represented by 73 1, is the portion of the waveform immediately after 
the sampled point 719 which includes spurious response points 721, 723, 725 and 727. The 
portion of the waveform 719 represents the spurious response of the channel. The adaptive 
transversal filter 623 makes a copy of the spurious response and subtracts it from the overall 
channel response in summation unit 619. The adaptive transversal filter is termed adaptive 

30 because it must adapt itself to the characteristics of the channel. In other words, the adaptive 
transversal filter must be trained using the channel characteristics in order to derive the proper 
response that will be provided to the summation unit 619. Once the slicer 621 detects which 
symbol is present, the adaptive transversal filter can then provide the response necessary to cancel 
the intersymbol interference present that would accompany the transmitted point. 

35 The received waveform at a point prior to the summation unit 619 is shown on 

oscilloscope 6 1 5 as display 6 1 5 A. Display 6 1 5 A is a scatter type waveform that does not exhibit 
distinct levels. The waveform 631 may be generated by the adaptive transversal filter 61 1, in 
order to cancel the intersymbol interference (ESI) within the channel. 
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1 Once the intersymbol interference is subtracted from the incoming signal in summation 

unit 619, the output of the summation unit appears as shown on oscilloscope 617, in display 
61 7 A. Display 617 A represents an eye diagram having five discrete levels. Once the levels have 
been well defined, as seen on display 61 7A, the sheer 621 is able to distinguish relatively easily 

5 between the symbols. The adaptive transversal filter 623 will respond to whatever symbol is 
found by the sheer 623 and provide the necessary waveform to cancel the intersymbol 
interference caused by the found symbol's transmission. The transversal filter generating an 
intersymbol interference replica, which must be subtracted from the incoming signal. The 
intersymbol interference waveform changes, depending on which symbol has been found by the 

1 0 sheer. The output of the decision feedback equalizer depends on the previously decoded 
symbols. The PAM-5 symbols found are then decoded by the Physical Coding Sublayer (PCS) 
521 and then provided to an interface such as a XGMII interface (not shown). 

To further set forth the inventive concepts, a preliminary simulation study of a PAM-5 
system is discussed. To simulate the PAM-5 system a laser model was created using rate 

1 5 equations. Pseudo-random PAM-5 data was introduced to the laser model. The output from the 
laser model was provided to fiber model . The fiber modeled was a multimode type fiber modeled 
as a linear system with Gaussian Impulse Response. In one example a 62.5/1 25 \im fiber having 
a bandwidth of 1 GHZ conveys a loss nanometer signal. 

The Nyquist theorem establishes that the bandwidth needed to transmit data at a rate F b 

20 = 1/T, without intersymbol interference, must be larger than or equal to l A T. Many 
communication systems however, signal at rates faster than 0.5 F b , using special techniques to 
control the intersymbol interference. Such techniques have been used in the 1 OOBase-TX and 
lOOOBase-T Ethernet transceivers. The present model examines signaling at 5 GBaud over 
multimode fibers with 1 GHZ bandwidth. The present simulations contemplate signaling at 2.5 

25 times the Nyquist rate (data rate equals 10 Gb/s, baud rate F b = 5 GHZ, bandwidth equals 1 
GHZ). This bandwidth assumption is consistent with 500 meters of 160/500 MHz-KM fiber at 
1350 nanometers (nm), or 160 meters of the same fiber at 850 nm, 

The model also assumes a receiver having a DFE as illustrated in Figure 6. It is 
recognized that equivalent equalization can be accomplished at the transmitter through the use 

30 of THP, DLP or other equalization techniques. 

The rate equations used in modeling the laser are described in "On Approximate 
Analytical Solutions of the Rate Equations for Studying Transient Spectra of Injection Lasers 5 ', 
by D. Marcuse and TJP, Lee s IEEE Journal of Quantrum Electronics, September 1983. The 
equations of the computer model are solved numerically using a fourth order Runge-Kutta 

35 Algorithm. The bias current in the equations was set to three times I^shou- ^ addition, a 6dB 
extinction ratio is used. 

Figure 8 is a block diagram illustrating a channel model used in conjunction with the rate 
equations. In Figure 8, a pseudo-random PAM-5 symbol generator S01 provides symbols to the 
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1 laser model 803 . The output of the laser model is then provided to a multimode fiber model 805. 
The fiber model 805 is a linear system having a Gaussian impulse response. 

Figure 9 is a graphic representing the impulse response of a linear system having a 
Gaussian impulse response convolved with a single pole high pass filter with corner frequency 

5 of 200 MHz. For the simulation, the fiber channel is modeled as in an article "Equalization of 
Multimode Optical Fiber Systems," by B.L. Kaspers, Bell Systems Technical Journal, September 
1982. The Kaspers' model comprises a linear dispersive system with a Gaussian impulse 
response given by equation 1 . Such Gaussian impulse response models are common throughout 
the literature and are considered to be fairly accurate for fibers in which all modes are excited 

1 0 equally (i.e. multimode fibers in an overfill launch condition). 

1 r,2j/i/.T\2 



25 



Kt)= ^ • 



15 (Equation 1) 

Where T is the period and a is a system dependent variable related to the bandwidth of the fiber. 
The corresponding frequency response of the channel is given by equation 2. 

H(f)^e~ {{l7taTJ)lfl] 

20 (Equation 2) 

The 3 dB bandwidth of the fiber is given by a equation 8 for a given baud period equal to 300 
pico seconds. 



_ 0.1325 

JldB ~ 



aT 

(Equation 3) 



If the 3 dB bandwidth of the system is assumed to be one gigahertz, the value of ^ is equal to 
30 0.6625. 

Figure 10 is a graph illustrating the output of a laser model based on the rate equations. 
The vertical axis 1001 represents the intensity of the laser. 

As can be seen from Figure 10, the signal appears to be somewhat noisy. The noise is 
partially accounted for by relaxation oscillation of the laser. Each time there is a sharp transition 
35 in the laser signal, overshoot and ringing results, as depicted in the graph of Figure 10. 

Figure 1 1 is a magnified portion of the graph of Figure 10. Figure 1 1 is included to 
illustrate the ringing present within the waveform of Figure 10. 

Figure 1 2 is a graph of the waveform of Figure 1 0 after passage through the fiber channel 
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1 model. In other words, Figure 12 is an illustration of the waveform presented to the receiver. 
As can be readily appreciated by observing Figures 10 and 12, the fiber channel performs 
significant filtering on the output of the laser. In other words, Figure 12 is a convolution of the 
waveform in Figure 10 with the impulse response of the channel as illustrated in Figure 9. 

5 Figure 13 is an eye diagram of a simulated equalizer at the transition between a training 

mode and receiving PAM-5 symbols. 

The simulated signal of Figure 12 is introduced to a receiver in Figure 13. The receiver 
used is one such as illustrated in Figure 6. The eye diagram depicted in Figure 13 is the output 
of the decision feedback equalizer 625 of Figure 6. The portion of the graph in Figure 13 

1 0 delineated by 1 3 0 1 represents a scatter diagram equivalent to the display 6 1 5(a) of Figure 6. The 
scatter diagram results after data has begun entering the receiver and being processed by the sheer 
but the equalizer has not yet been trained. In other words, the adaptive transversal filter has not 
yet adapted to the characteristics of the channel. A portion of the graph illustrated at 1303 in 
Figure 1 3 is an illustration of the equalizer being trained using two level symbols. At point 1315, 

1 5 within Figure 13, the five level Pam alphabet is transmitted to the receiver. The results of the 
receiving of the five level Pam -5 alphabet in the trained decision feedback equalizer 625 is the 
five levels 1305, 1307, 1309, 131 1, and 1313 depicted in Figure 13. Figure 14 is a graph 

of a magnified portion of the graph of Figure 13, illustrating the convergence of the equalizer 
during a training sequence. 

20 Figure 14 is a time expansion of the section 1301 of Figure 13. As can be seen from 

Figures 1 3 and 1 4, high data rates can be achieved using pulse amplitude modulation despite the 
presence of inner symbol interference, which may result from signaling faster than the Nyquist 
rate. 

As can be seen from the models used to simulate the PAM-5 system transmitting data a 
25 faster than the Nyquist rate is viable using the techniques disclosed. Additionally using 
multilevel symbols has been shown to be viable and may be used to increase the data rate across 
fiber channels. The reduced noise tolerance of the system due to the reduced distance between 
signaling levels of a multilevel signaling system may be counteracted by the use of trellis coding 
the signals transmitted. 

30 Supplemental material with regard to the invention here and above described in Appendix 

A entitled "lOGb/s PMD Using PAM-5 Trellis Coded Modulation". 



35 



-13- 



SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



WHAT IS CLAIMED IS: 

1 . A method for high-speed transmission of information data on an optical channel, 
the method comprising : 

encoding information via a trellis encoder to produce digital multilevel symbols; 
converting the digital multilevel symbols into analog multilevel signals; and 
transmitting the analog multilevel symbols over an optical channel. 

2. The method of claim 1 further comprising equalizing the digital multilevel symbols 
to compensate for characteristics of the optical channel. 

3. The method of claim 1 further comprising of equalizing the analog multilevel 
symbols to compensate for characteristics of the optical channel. 

4. The method of claim 2 wherein equalizing the digital multilevel symbols comprises 
precoding the digital multilevel symbols using a Tomlinson Harashima precoder. 

5. The method of claim 2 wherein the equalizing the digital multilevel symbols 
comprises precoding the digital multilevel symbols using a dynamic limiting precoder. 

6. The method of claim 1 wherein the converting the digital multilevel symbols to 
analog multilevel symbols includes mapping the digital multilevel symbols into a subset mapper. 

7. The method of claim 1 wherein transmitting the analog multilevel symbols over an 
optical channel comprises modulating the intensity of a light source according to the level of the 
analog multilevel symbols. 

8 . The method of claim 1 wherein transmitting the analog multilevel symbols over an 
optical channel comprises modulating laser intensity according to a level of the analog multilevel 
symbols. 

9. A method as in claim 2 wherein equalizing the digital multilevel symbols to 
compensate for the laser and channel characteristics comprises: 

characterizing the channel; and 

applying an inverse characterization of the channel to the digital multilevel 
symbols; 

10. A method as in claim 2 wherein equalizing the digital multilevel symbols to 
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1 compensate for the laser and channel characteristics comprises: 

characterizing the channel; and 

applying an inverse characterization of the channel to the analog multilevel 
symbols. 

5 

11. A method for high speed transmission on an optical channel, the method 
comprising; 

accepting information from a plurality of sources; 

encoding the information via a plurality of trellis encoders to produce a plurality 
10 of digital multilevel symbols; 

converting the plurality of digital multilevel symbols into a plurality of analog 
multilevel signals; and 

transmitting the analog multilevel signal by time division multiplexing the plurality 
of analog multilevel signals onto an optical channel. 

15 

12. A method as in claim 1 1 wherein converting the plurality of digital multilevel 
symbols to analog multilevel signals further comprises: 

mapping the digital multilevel symbols in a subset mapper; and 
forming analog multilevel symbols by providing analog representations of mapped 
20 multilevel signals. 

13. The method of claim 11 further comprising equalizing the plurality of digital 
multilevel symbols to compensate for characteristics of the optical channel. 

25 14. The method of claim 11 further comprising equalizing the plurality of analog 

multilevel symbols to compensate for characteristics of the optical channel. 

15. The method of claim 13 wherein equalizing the digital multilevel symbols 
comprises precoding the plurality of digital multilevel symbols using a Tomlinson Harashima 

30 precoder. 

16. The method of claim 12 wherein equalizing the digital multilevel symbols 
comprises precoding the plurality of digital multilevel symbols using a dynamic limiting 
precoder. 



35 



17. The method of claim 11 wherein converting the plurality of digital multilevel 
symbols into a plurality of analog multilevel symbols further comprises: 

accepting the plurality digital multilevel symbols into a subset mapper; and 
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1 forming a plurality of mapped analog multilevel symbols from the plurality of 

digital multilevel symbols. 

18. The method of claim 1 1 wherein transmitting the plurality of analog multilevel 
5 symbols over an optical channel comprises modulating the intensity of a light source according 

to the levels of the plurality of analog multilevel symbols. 

19. The method of claim 1 1 wherein transmitting the plurality of analog multilevel 
symbols over an optical channel comprises modulating the intensity of a laser according to the 

1 0 level of the analog multilevel symbols. 

20. The method as in claim 12 wherein equalizing the digital multilevel symbols to 
compensate for the laser and channel characteristics comprises: 

characterizing the channel; and 
15 applying an inverse characterization of the channel to the digital multilevel 

symbols. 

21. . The method as in claim 12 wherein equalizing the digital multilevel symbols to 
compensate for the laser and channel characteristics comprises: 

20 characterizing the channel; and 

applying an inverse characterization of the channel to the plurality of analog 
multilevel symbols. 

22. The method as in claim 1 1 wherein converting the plurality of digital multilevel 
25 symbols into a plurality of analog multilevel signals comprises: 

accepting the plurality digital multilevel symbols successively into a single analog 
to digital converter; and 

successively converting the plurality of symbols into analog multilevel symbols. 

30 23 . The method as in claim 1 1 wherein converting the plurality of digital multilevel 

symbols into a plurality of analog multilevel signals comprises: 

accepting the digital multilevel symbols successively into a plurality of analog to 
digital converters; 

converting the plurality of symbols into an analog representation: and 
35 successively combining the analog multilevel symbols into a succession of analog 

multilevel symbols, 

24. A method of receiving data from an optical channel the method comprising: 
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1 accepting a multilevel signal from the channel into an optical to electrical 

converter; 

converting the multilevel signal into an analog electrical signal; 
converting the analog electrical signal into a digital signal; and 
5 decoding the digital signal in a trellis decoder. 

25. The method of claim 24 further comprising equalizing the digital signal prior to 
decoding the digital signal in the trellis decoder. 

1 0 26, The method of claim 25 wherein equalizing the digital signal comprises applying 

a decision feedback equalization to the digital signal. 

27. A method as in claim 24 wherein converting the analog electrical signal to a digital 
signal comprises: 

1 5 successively sampling the analog electrical signal; and 

converting the successive samplings into a plurality of parallel digital values. 

28. A method of signaling over an optical channel the method comprising: 
accepting data from a source; 

20 trellis encoding the data; 

coupling the encoded data into an optical channel; 

conveying the data over the optical channel; 

accepting data from the optical channel 

decoding the data accepted from the optical channel; and 
25 providing the decoded data to an interface. 

29. A method as in claim 28 further comprising: 
equalizing the data after trellis encoding the data. 

30. A method as in claim 29 wherein equalizing the data comprises applying a 
30 Tomlinson-Harashima precoding to the data. 

31. A method as in claim 3 0 wherein equalizing the data comprises applying a dynamic 
limited precoding. 

35 32. An apparatus for transmitting information on an optical channel to apparatus 

comprising: 

a trellis encoder for accepting digital information and producing digital multilevel 
signals; 
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a digital to analog converter that accepts the digital multilevel signals and produces 
analog multilevel signals; and 

an analog signal to optical converter that converts the analog signal to an optical 
level for coupling into an optical channel 

33 . The apparatus of claim 32 further comprising an equalizer that accepts the digital 
multilevel signals and produces equalized digital multilevel signals prior to coupling into the 
digital to analog. 

34. The apparatus of claim 32 further comprising an equalizer that accepts the analog 
multilevel signals and produces equalized analog multilevel signals. 

35. Aji apparatus as in claim 33 wherein the equalizer is a Tomlinson-Harashima 
precoder. 

36. An apparatus as in claim 33 wherein the equalizer is a dynamic limiting precoder. 

37. An apparatus as in claim 32 wherein the analog signal to optical level converter 
includes a laser. 

38. An apparatus for concurrently transmitting a plurality of data signals over an optical 
channel, the apparatus comprising: 

a plurality of trellis encoders that accept a plurality of data signals and produce a 
plurality of digital multilevel signals; 

a converter that accepts a plurality of digital multilevel signals and produce a 
plurality of analog multilevel signals; 

an optical source that receives the plurality of analog multilevel signals and 
produces a light output proportional to the level of successive analog multilevel signals 
for driving an optical channel. 

39. An apparatus as in claim 3 8 further comprising a plurality of equalizers that accept 
the plurality of digital multilevel signals and produce a plurality of equalized digital multilevel 
signals to provide to the converter. 

40. An apparatus as in claim 39 wherein the plurality of equalizers comprise a plurality 
of Tomlinson-Harashima precoders. 

41. An apparatus as in claim 39 wherein the plurality of equalizers comprise a plurality 
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1 of dynamic limiting precoders. 

42. An apparatus for concurrently transmitting aplurality of data signals over an optical 
channel the apparatus comprising: 

5 a plurality of trellis encoders that accept a plurality of data signals and produce a 

plurality of digital multilevel signals; 

an analog to digital converter that sequentially accepts the plurality of digital 
multilevel signals and produces a plurality of sequential analog multilevel- signals; 

an optical source that receives the plurality of analog multilevel signals for driving 
10 an optical channel. 

43 . An apparatus as in claim 3 8 further comprising a plurality of equalizers that accept 
the plurality of digital multilevel signals and produce a plurality of digital multilevel signals. 

15 44. An apparatus as in claim 43 wherein the plurality of equalizers comprise aplurality 

of Tomlinson-Harashima precoders. 

45 . An apparatus as in claim 43 wherein the plurality of equalizers comprise a plurality 
of dynamic limiting precoders. 

20 

46. An apparatus for receiving data from an optical channel the apparatus comprising: 
an optical to electrical converter for receiving an optical multilevel signal from an 

optical channel and converting the optical multilevel signal into an analog multilevel 
electrical signal; 

25 a decoder that accepts the analog multilevel electrical signal and converts it into 

digital multilevel signal; 

a trellis decoder that accepts and decodes the digital multilevel signal producing 

data. 

47 . The apparatus of claim 46 further comprising an equalizer for accepting the digital 
30 multilevel signal and producing a digital equalized multilevel signal for coupling into the trellis 

decoder. 

48. An apparatus as in 47 wherein the equalizer is a decision feedback equalizer. 

35 



-19- 



SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



1/15 



< 

o 

on 

Q_ 




if) 



Ld 
CO 




SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/USO 1/01532 



2/15 



15 



CD 
O 



O 

o 

CM 



o cc 

__> °. £ 



__ _TTT— 





*8 



01 



CO 

z: 
< 




SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



3/15 




SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



4/15 



CL 



O 
Z 

Q 
O 
U 
UJ 
QsL 
CL 

^ < 

I 

z 

o 

CO 



o 




Ld 












< 


+ 






CJ 






O 



cn 
o 



LjJ 








< 




I 




o 





o 



tn 



O 
C 

o 

CD 
CO 



CO 



o 
o 



o 

E 

O 
O 



CD 



or 

UJ 
Q 
O 



O LlJ 




SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 PCT/US01/01532 



5/15 




SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



6/15 



I 



UJ 
h- 

CO 

>- 
CO 



o 



-1 



r 



to 



CM 
LO 



ji — i. \ — — * 



CO 

o 

CL 



LO 



LO 



O 



en 



CO 



LxJ o 

I 



S7 



zoo 



LiJO 0 ^ 
Lu<2 



UJO . 
?00 

u-<z: 




o S 

I UJ 




LO 
LO 



SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



7/15 



< 




SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



8/15 



UJ 




SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



9/15 



GO 



LU 

o 

o 



LU 
< 




O 
00 



< o 



W q Q Q! h 

^ O w < 
(/) U 3 

< X Qf 



o 

00 



o 

CD 



o 



o 

Q O 

ZD d 

0- Or: ^ 00 jfj 

o 



< 

LU 



SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



10/15 



O 




SUBSTITUTE SHEET (RULE 26) 




SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



12/15 




SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 




SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



14/15 




O 



SUBSTITUTE SHEET (RULE 26) 



WO 01/54317 



PCT/US01/01532 



15/15 




SUBSTITUTE SHEET (RULE 26) 



(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 

International Bureau 

(43) International Publication Date 
26 July 2001 (26.07,2001) 






III 




PCT 



(10) International Publication Number 

WO 01/54317 A3 



(51) International Patent Classification 7 : H04L 25/03, 

25/497, 27/02 

(21) International Application Number: PCT/US01/01532 

(22) International Filing Date: 17 January 2001 (17.01.2001) 

(25) Filing Language: English 

(26) Publication Language: English 



(30) Priority Data: 

60/177,034 



17 January 2000 (17.01.2000) US 



(71) Applicant (for all designated States except US): BROAD- 
COM CORPORATION [US/US]; 16215 Alton Parkway, 
Irvine, CA 92618 (US). 

(72) Inventor; and 

(75) Inventor/Applicant (for US only): AGAZZI, Oscar, E. 
[US/US]; 16215 Alton Parkway, Irvine. CA 92618 (US). 



(81) Designated States (national): AE, AG, AL, AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CR, CU, CZ, 
DE, DK, DM, DZ,EE, ES, FI, GB, GD, GE, GH, GM, HR, 
HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, LK, LR, 
LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, MZ, 
NO, NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SL, TJ, TM, 
TR, TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZW) T Eurasian 
patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), European 
patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE 
IT, LU, MC, NL, PT, SE, TR), OAPI patent (BE BJ, CF, 
CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG). 

Published: 

— with international search report 

(88) Date of publication of the international search report: 

7 March 2002 



(74) Agent: WINSLADE, Christopher, C; McAndrews For two-letter codes and other abbreviations, refer to the "Guid- 
Held & Malloy, Ltd., 500 W. Madison Street, Suite 3400, once Notes on Codes and Abbreviations" appearing at the begin- 



Chicago, EL 60661 (US). 



rang of each regular issue of the PCT Gazette. 



(54) Title: TRELLIS-CODING AND PRE-EQUALISATION FOR OPTICAL CHANNELS 



< 



TT 



TRANSMITTER 



200 



r 



R BTS< 



r ^-307 ~ 1 



R-M UNENCODED BITS 



' r 


305 


1 

J 


SUBSET 








MAPPER 


'm BITSl 


RATE 
M/(M+1) 
CONV.CODER 








M+1 BITS 





TRELLIS ENCODING— 303 



r 



309 



~i 



* 



T0MLINS0N 
PREC0DER 



311 



L 



209 




^? (57) Abstract: A method and apparatus for transmission of data on bandwidth limited fiber optic channels. A multilevel signaling 
alphabet having multiple levels of optical intensity are used to transmit signals on optical channels. In order to counteract the decrease 
Q in signal to noise ratio resulting from the use of a multilevel signaling alphabet over a bilevel signaling alphabet trellis encoding of the 
^ data to be transmitted is employed. To counteract intersymbol interference due to signaling faster than the Nyquist Rate, equalization 
^ methods such as Tomlinson-Harashima precoding and decision feedback equalization are employed. 



INTERNATIONAL SEARCH REPORT 



ational Application No 

PCi/US 01/01532 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC 7 H04L25/03 H04L25/497 H04L27/02 



According to International Patent Classification (IPC) or to both national classification and IPC 



8. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

IPC 7 H04L 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during ihe international search (name of data base and, where practical, search lerms used) 

PAJ, WPI Data, EPO-Internal , INSPEC, COMPENDEX 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category • 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



EP 0 675 622 A (FUJITSU) 
4 October 1995 (1995-10-04) 
page 2, line 10 
page 2, line 50 - line 58 

DE 44 40 947 A (K0MMUNIKATI0NS ELEKTRONIK) 
25 April 1996 (1996-04-25) 



1-48 



1-48 



column 


1, 


line 


19 - 


line 


21 


column 


1, 


line 


64 - 


1 ine 


68 


column 


2, 


line 


45 - 


1 ine 


53 



WO 98 39871 A (LEVEL ONE COMMUNICATIONS) 

11 September 1998 (1998-09-11) 

page 1, line 7 

page 1, line 21 - line 22 

page 3, line 13 - line 19 

-/- 



1-48 



Further documents are listed in the continuation of box C. 



Patent family members are listed in annex. 



• Special categories of cited documents : 

•A' document defining Ihe general state ot the art which is not 
considered to be of particular relevance 

'E* earlier document but published on or after ihe international 
filing date 

•|_* document which may throw doubts on priority claim(s) or 
which is cited lo establish the publication dale of another 
citation or other special reason (as specified) 

'O* document referring to an oral disclosure, use. exhibition or 
other means 

'P* document published prior to the international tiling date but 
later than the priority date claimed 



'T* later document published after the international filing date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

*X* document of particular relevance; Ihe claimed invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when Ihe document is taken alone 

*Y* document of particular relevance; the claimed invention 
cannot be considered to involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 

document member of the same patent family 



Date of ihe actual completion ot the international search 



17 July 2001 



Date ot mailing ot the international search report 

30/07/2001 



Name and mailing address of the ISA 

European Patent Office, P.B. 5818 Patent laan 2 
NL - 2280 HV Rifswtjk 
Tel. (+31-70)340-2040. Tx. 31 651 epo nl, 
Fax: (+31-70) 340-3016 



Authorized officer 



Scriven, P 



Form PCT/1SA/210 (second shwt) (July 1992) 



nanp 1 of ? 



INTERNATIONAL SEARCH REPORT 



lr -ational Application No 

PtT/US 01/01532 



C(Continuation) DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document, with indication .where appropriate, of the relevant passages 



Relevant to claim No. 



UYEMATSU ET AL.: "Trellis coded 
modulation for multilevel photon 
communication system" 
ICCS/ISITA 1992, 

16 - 20 November 1992, pages 582-587, 
XP002166368 
New York, US 
ISBN: 0-7803-0803-4 

page 582, right-hand column, paragraph 2 



PI-YAN6 CHIANG ET AL. : "Using precodlng 
technique to reduce the BER penalty of an 
M-QAM channel in hybrid AM-VSB/M-QAM 
subcarrier multiplexed lightwave systems" 
IEEE PHOTONICS TECHNOLOGY LETTERS, 
vol. 10, no. 8, August 1998 (1998-08), 
pages 1177-1179, XP000769893 
New York, US 
ISSN: 1041-1135 

page 1177, right-hand column, paragraph 2 
- paragraph 3 



1-3, 

6-14, 

17-29, 

32-34, 

37-39, 

42,43, 

46-48 



4,5,15, 

16,30, 

31,35, 

36,40, 

41,44,45 

4,5,15, 

16,30, 

31,35, 

36,40, 

41,44,45 



Form PCT/ISA/210 (continuation ot second sheet) (July 1992} 



oaae 2 of Z 



INTERNATIONAL SEARCH REPORT 

Information on patent family members 



Patent document 
cited in search report 



EP 0675622 



Publication 
date 



|r national Application No 

PCf/US 01/01532 



Patent (amity 
member(s) 



04-10-1995 



JP 
US 
US 
US 
US 



7273827 A 
5734681 A 
6104758 A 
6026122 A 
6009123 A 



Publication 
date 



20-10-1995 
31-03-1998 
15-08-2000 
15-02-2000 
28-12-1999 



DE 4440947 


A 


25-04-1996 


US 


5854812 A 


29-12-1998 


WO 9839871 


A 


11-09-1998 


US 


6167082 A 


26-12-2000 








AU 


6685098 A 


22-09-1998 








EP . 


0965207 A 


22-12-1999 



Form PCT/lSA/210 (potent lamify armex) (July 1902) 



